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ABSTRACT 



The reasons for the deterioration of the inspection grades 
awarded for work-based learning (WBL) provision in England were examined. The 
main data collection activities were as follows: (1) a statistical analysis 

of the Training Standards Council (TSC) and Adult Learning Inspectorate (ALI) 
inspection grades in 1998-2001; (2) a qualitative review of inspection data; 

and (3) structured interviews with ALI inspectors, Learning and Skills 
Council quality managers, and WBL providers. All inspections were revealing 
significant problems in the quality of WBL, and the proportion of poor grades 
was increasing. The increase in poor grades awarded for WBL was linked partly 
to the transition to the new Common Inspection Framework (CIF) and Modern 
Apprenticeships frameworks and new arrangements for quality assurance and 
funding of post- 16 education and training. Other reasons were related to 
deeper systemic issues. The following were among the study recommendations: 

(1) seek greater clarity and transparency in expectations about success rates 
and better and more comprehensive guidance on the CIF; (2) ensure that the 
range of programs offered is broad enough to encourage participation, sustain 
learners' progression, and secure progression; (3) build the capacity of WBL 
providers and develop new ways of delivering WBL; and (4) improve achievement 
in Modern Apprenticeships. (Contains 16 tables/figures.) (MN) 
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Summary 



Background 

The Learning and Skills Development Agency (LSDA) has been investigating the 
reasons for the deterioration in the inspection grades awarded for work-based 
learning provision. The changing grade profile is a continuation of a trend which 
began in inspections conducted by the Training Standards Council (TSC) in 
its last year of operation. The trend increased in grades awarded by the Adult 
Learning Inspectorate (ALI) in the first nine months since it was established. 

The research involved : 

■ a statistical analysis of the TSC and ALI inspection grades from 1998 to 2001 

■ a qualitative review of inspection data looking at strengths and weaknesses 
identified in ALI inspection reports 

■ structured interviews with ALI inspectors, Learning and Skills Council (LSC) 
quality managers and work-based learning providers. 

Because there have been significant changes to the framework within which 
inspection is conducted and, to some extent, to the programmes within 
government supported work-based learning, such as Modern Apprenticeships, 
it was not possible to draw direct comparisons over the period concerned. 
Rather the focus of the research was on the critical issues which are important 
for the development of better quality within work-based learning. 



Main findings 

■ ALI inspections are revealing significant problems in the quality of work-based 
learning. The proportion of poor grades increased in the last year of operation 
of the TSC and increased further in subsequent ALI inspections. A noticeable 
deterioration in the last year of operation of the TSC has accelerated under ALI. 

■ There appears to be no single cause for the poor grades awarded for work-based 
learning. Some reasons appear to be linked to the transition into new arrangements 
for inspection, quality assurance and funding arrangements of post-16 education 
and training, while other reasons may be related to deeper systemic issues. 



Implications of findings 



Transitional issues affecting quality and standards 

■ Inspection is a relatively new process for work-based learning providers. 
Lack of familiarity with the process of inspection coupled with changes to 
the inspection framework may have contributed to a lack of preparedness 
of providers inspected early in the ALI inspection cycle. 
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The transition from Training and Enterprise Councils (TECs ) to local LSCs 
may have reduced the support available to providers who were preparing 
for inspection in the early days of the LSC’s operation. 

Providers have problems with the availability, collection and use of data 
which is needed to provide evidence of learners’ achievement. 

Many of these transitional issues have now been resolved or are actively 
being addressed. 



Systemic issues 



Better work-based learning 

The Common Inspection Framework (CIF) emphasises the benefit to the 
individual learner as the principal criterion forjudgements on the quality 
of provision. Modern Apprenticeship frameworks require greater theoretical 
understanding and the development and assessment of key skills. 

These developments have switched the emphasis from the assessment of 
competence in a work role to the development and application of knowledge 
and skills in a work context. The inspection process is therefore looking for 
a wider range of learning activities and observable learning in the workplace. 

Providers need more support to come to terms with these developments and 
to adapt their methods of teaching, learning and assessment accordingly. 



Success in work-based learning 

The Common Inspection Framework (CIF) places considerable emphasis 
on learner success in achieving the aims of the apprenticeship or other 
work-based learning. Success rates within work-based learning are improving 
but are currently low, and this is a contributory factor in the proportion of 
low grades that are awarded. There are, however, concerns about how these 
measures of success capture the labour market context of work-based learning 
for young people. 



Building capacity in the work-based training sector 

There is a low proportion of staff with relevant qualifications in the 
work-based sector. 

There has been little sustained and systematic development of staff. Demands 
for more rigorous learning and assessment in the work-based route need to be 
accompanied by support and development of the people charged with its delivery. 

The work-based sector needs to be supported to take a strategic view of its 
own development through better self-assessment and quality improvement. 



Range of provision 

The range of programmes available for young people may not be meeting the needs 
of all learners in work-based learning. As a result, learners may be placed on 
inappropriate programmes, which they subsequently leave without completing. 

There are important related issues of good careers guidance and accurate 
initial assessment followed by the delivery of effective learner support. 



part a 

analysis of key issues 



Introduction 



The Learning and Skills Development Agency (LSDA) was requested by the 
Department for Education and Skills (DfES) to investigate the reasons for 
the deterioration in the grades awarded to work-based learning provision from 
Adult Learning Inspectorate (ALI) inspections compared with Training Standards 
Council (TSC) inspections. The change is a continuation of a downward trend 
from the last year of TSC inspections, but has sharply accelerated since 
the beginning of April to the end of December 2001. In contrast, grades for 
New Deal provision, which is also inspected by the ALI, continue to improve. 

The LSDA investigation into the underlying reasons for the changes in the grade 
profile from TSC to ALI indicates that some of the reasons may be transitional and 
related to the changes in post-16 inspection and quality assurance arrangements. 
These shortcomings may be easily remedied in the short term as all concerned 
become accustomed to new ways of working. However, there may also be deeper 
systemic problems outside the inspection framework which require attention. 



Declining inspection grades under ALI 

A statistical analysis of ALI inspection grades for work-based learning confirms 
the continuation of the trend for poorer grades which was evident in later TSC 
inspections. The changes to the inspection framework and varied nature of 
the provider base and learners who participate in work-based learning make 
comparing like with like an impossible task and so care must be taken in any 
conclusions derived from this evidence alone. The ALI data considered in this 
report also covers less than a full year, and consequently there are some small 
numbers within the subgroups that have been examined. That said, the average 
area of learning (AoL) grade 1 has deteriorated from 2.7 2 in 1998/9 to 3.32 in 
2001/02 and the average leadership and management grade has worsened from 
2.97 to 3.59 in the same period. While there was a steady downward trend under 
the TSC, there has been a marked deterioration since the beginning of ALI. 

ALI grades awarded for New Deal, which is also inspected with the Common 
Inspection Framework (CIF), have improved from those awarded by the TSC. 



Key issues 

Transitional issues affecting quality and standards 

Turbulence and uncertainty in the period leading up to and immediately 
following the inception of the Learning and Skills Council (LSC) appears to have 
had a significant impact on providers’ readiness for inspection. The research 
revealed some instances of providers preparing their self-assessment report 
(SAR) against the former inspection criteria given in Raising the Standard. 3 



1 Average grades are used in the report as an indicative measure of change. As the gaps between grades 
are not equally spaced, percentages of grades 1 and 2 and 4 and 5 are also given in some cases. 

2 Grades in the Common Inspection Framework (CIF) range from lto 5, where grade 1 is outstanding 
and grade 5 is very weak. 

3 TSC. Raising the Standard: guidelines for self-assessment and inspection of work-based training . 
Training Standards Council, April 1998. 
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While the primary responsibility for improving the quality of provision must rest 
with providers, local LSC quality managers themselves observed that support 
to providers from some local LSCs was not as knowledgeable or sufficient 
as it should have been. Most of the providers inspected by the ALI were 
not inspected by theTSC, and the process is therefore entirely new to them. 

Providers are often unsure about what is required and how to amend their 
practice to secure the levels of quality which are now expected. However, 
access to the Standards Fund could enable providers to gain support to 
improve. Indeed, some providers reported that they were able to remedy 
most of the additional administrative and process-related deficiencies in 
their provision after these had been revealed by their inspection. 

There is evidence of improving support as LSC staff gain experience in their 
new roles. Providers who have attended the ALI briefing sessions say these 
are helpful in understanding the CIF and inspection process. Support for 
preparation could be further improved if the CIF was accompanied by more 
detailed guidance on preparing for inspection and the self-assessment process. 

Providers are not good at collecting and analysing data to obtain information they 
could use to improve their performance. They have difficulty in understanding 
the way that data is used in the inspection process. Greater transparency in 
what data is being used and its purpose, together with training for providers 
in the use of data, could result in quick wins in quality improvement. 

Systemic issues 

The decline in the grade profile for work-based learning is too stark to be 
entirely explained by the turbulence caused by the introduction of the new 
post-16 arrangements. The marked deterioration in the grade profile in the 
20 providers that have experienced both ALI and TSC inspections suggests 
that there are significant underlying issues. These issues may be related to 
the capacity of providers to deliver the more rigorous Modern Apprenticeship 
programmes and the increased scrutiny of the CIF, with its emphasis on the 
impact of provision on learning. 



Triangulating concerns 

The research identified the different perspectives of the key players on 
the nature of the problem. 

Providers : 

■ are aware of the impact of retention and achievement, but do not link this 
to impact on learners and learning 

■ are working with a new system of inspection which they do not fully understand 

■ lack support or resources to remedy their shortcomings 

■ have difficulty in delivering key skills 

■ think they are working with increasingly less able or motivated trainees 

■ have problems in obtaining and using data. 
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Analysis of key issues 3 



LSC quality managers: 

■ have significant concerns about the impact of provision on learners and learning 

■ see data collection and interpretation as a key issue 

■ see providers as relatively immature organisations 

■ see key skills and training programme issues as part of the problem 

■ share to some extent the providers’ views about changes to the cohort 
of learners in the work-based route. 

ALI inspectors: 

■ closely link retention and achievement issues with impact on learners and learning 

■ share a similar priority ranking with providers regarding lack of support, 
and to a lesser degree with providers’ unfamiliarity with the CIF 

■ agree that there are key skills and other programme-related issues 

■ think there is some relationship to the size and scale of the organisation. 



Support for changes to Modern Apprenticeships 

The provision inspected since the ALI was set up has been predominantly 
Foundation and Advanced Modern Apprenticeships, these generally being 
undertaken by young people who are entering the workforce for the first time. 
This reflects both the shift of responsibilities for work-based learning for 
adults to the Employment Service (ES) and the phasing out of ‘other training’. 
Many of the Modern Apprenticeship frameworks have recently been revised. 

All require the successful achievement of a whole Level 2 or 3 NVQ and key skills. 
Some frameworks include additional qualifications or units of qualifications. 
Achievement of a Modern Apprenticeship programme is therefore more complex 
than achieving an NVQ, requiring in most circumstances dedicated time for 
off-the-job learning. 

Key skills are almost universally seen by providers as a problem, and 
irrelevant to the young people and their employers. The introduction of 
technical certificates is seen as a further potential problem. These changes 
are a major shift for providers of work-based learning, and the findings reveal 
that many providers are ill prepared. 

The emphasis on a wider range of learning activities and more structured delivery 
of underpinning theory within Modern Apprenticeships is revealing pedagogical 
issues. There is a need for more regular and planned ‘learning episodes’, within 
a work-related context. In some occupational areas there is no tradition of this 
type of learning and models for its delivery need to be developed. 



Focus on the impact of provision on learning 

The key question in the CIF asks, ‘How well do learners achieve?’ and this factor 
drives the inspection of quality. The primary evidence source for this is retention 
and achievement rates. The availability of more reliable (but not yet satisfactory) 
year-on-year data on retention and achievement has enabled the ALI to make 
more robust judgements. 

The reason that the focus on teaching and learning is having a negative impact on 
grades is related to the subsequent switch from assessment of competence in 
role to the development of knowledge and skills in a work context. The work-based 
sector is geared up to deliver the former and will need support to find new ways 
of delivering effective work-based learning for new entrants to the workforce. 
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The inspections considered in this research project have almost entirely 
focused on training for young people at a formative stage in their career. 

The picture may be different for adults who are mature learners, for whom 
a work-based assessment of competence may be entirely appropriate. 

These increasing demands appear to have taken providers by surprise, and 
support for their introduction was patchy acrossTraining and Enterprise Councils 
(TECs) and in the early work of the local LSCs.The expertise within the 
work-based route clearly has many strengths in terms of occupational expertise, 
and more recently in assessment processes. However, experience in more 
structured work-based teaching and learning is less well developed. 

Success in work-based learning 

Providers are concerned that the AU’s expectations of the success rate of 
learners on work-based learning provision are too high. They comment on 
the lack of a shared agreement on what should be the average retention 
and achievement rates for work-based learning provision, differentiated 
by programme and occupational area. 4 

Data collection in work-based learning has historically been poor and 
inconsistent. The shifting definition of successful completion from N VQs to 
full Modern Apprenticeship frameworks is also seen by providers to limit the 
extent to which they can achieve good inspection grades. As it is some years 
since the Modern Apprenticeships were introduced, more young people should 
have completed their training. Expectations of a good track record in successful 
completion have therefore increased. 

However, achievement within work-based learning is approximately 50% at best. 
Achievement rates - as measured by the percentage of trainees who leave 
with a full or part NVQ qualification - have varied considerably across the 
different government-supported training programmes. This relatively modest 
achievement is reflected in the grade profile of ALI inspections and is an 
indication of issues outside the inspection process which are impacting on 
the quality of provision. It is likely that the CIF and the ALI inspection process 
are merely reflecting these problems. 



Range of provision 

Providers and LSC quality managers raised concerns about the insufficient 
range of provision to meet the needs of all actual and potential learners in the 
work-based route. As a result, young people are often placed on inappropriate 
programmes, which they subsequently leave without completing. 

Some providers reported that they are being encouraged to place trainees 
on Advanced (rather than Foundation) Modern Apprenticeship programmes, 
which are not the best option for these particular young people. 

The absence of flexibility in programme design may therefore contribute 
to poor retention. This is a different, although related, issue from poor 
initial assessment. 



4 We understand that the ALI is working on benchmarking data. 
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Analysis of key issues 5 



Quality improvement lessons from New Deal 

While there are similarities in the delivery and inspection of New Deal and 
work-based training - in terms of the inspection framework, and in some of the 
people delivering and inspecting it - there are differences in programme design 
and expected outcomes. These differences may reveal some of the reasons for 
New Deal’s improving grade profile. Emphasis on job outcomes has been a 
consistent policy objective of New Deal. It also is, of necessity, more learner 
centred, with flexible and short-term goals, such as units of NVQs, being 
accepted as successful completion. More networking and exchange of 
good practice is evident, and collaboration between providers is encouraged. 
There has been good and sustained support locally from the unit of delivery 
coordinators and nationally from Employment Services. The support that 
providers receive and the more flexible view of achievement may make 
less onerous the task of delivering learning to a very difficult client group. 



Capacity building 

Many providers and LSC staff commented on the lack of sustained and 
sufficient development and training for staff within the work-based sector. The 
contrast with the college sector and its tradition for networking and exchanging 
practice was frequently drawn. Many of those interviewed commented on 
work-based trainers’ lack of access to the FE Standards Fund until very recently. 

There is a need to support a more critical, self-improving culture across all 
providers of work-based learning, together with a greater strategic awareness 
of how this contributes to their own and their learners’ success. 

The emphasis on a wider range of learning activities and more structured delivery 
of underpinning theory has significant implications for staff development and 
training. Providers will need support to amend their practice. The work-based 
training sector is under-qualified, even in terms of the previous focus on 
assessment. The changing emphasis on impact of learning has taken many 
by surprise and the cost of re-skilling assessors to become trainers is seen 
as prohibitive by many providers. 

Problems in delivering work-based learning are apparent across all types 
of provider, including college-based provision. This perhaps illustrates that 
work-based learning is difficult to deliver. While the workplace may provide a 
rich environment for learning, there are difficulties in securing the time, space 
and resources to learn as well as work. Support for the development of methods 
of work-based learning which work is needed for all types of providers. 



Funding and resourcing issues 

Some providers commented on the reduction in real terms of the money 
available to support work-based learning, and on problems with the standard 
length of stay. Meeting the increased demands of Modern Apprenticeship 
programmes within the available resources appears to be problematic. This is 
often further exacerbated by the modest entry qualifications of trainees, and 
their need for significant additional support and extra time on the programme 
to enable them to achieve. 
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1 Background 



The Learning and Skills Development Agency (LSDA) was requested by the 
Department for Education and Skills (DfES) to investigate the reasons for 
the deterioration in the grades awarded to work-based learning provision from 
Adult Learning Inspectorate (ALI) inspections compared with Training Standards 
Council (TSC) inspections. The change is a continuation of a downward trend 
from the last year of TSC inspections, but has sharply accelerated since 
the beginning of April to the end of December 2001. In contrast, grades for 
New Deal provision, which is also inspected by the ALI, continue to improve. 

Preliminary discussions between the ALI, the Learning and Skills Council (LSC) 
and the DfES, and an internal investigation of the views of inspectors by the ALI, 
identified possible causes. It was suggested that the sharp downturn in grades 
may be related to: 

■ changes to the framework and method of inspection 

■ changes in grade assessment, benchmarking and moderation methods 
among inspectors 

■ disruption caused by the transition from Training and Enterprise Councils (TECs ) 
to (local) LSC(s) 

■ changes in provider and employer actions and funding for improvement 

■ changing characteristics of learners and providers inspected. 

The project investigated the possible contribution of these causes, 
but also looked for other reasons. 

The research took place between December 2001 and February 2002. 



i.i Research question 

The research question addressed by the project was as follows : 

Why are grades awarded for work-based learning by the Adult Learning 
Inspectorate (ALI) substantially worse than those awarded by the 
Training Standards Council (TSC), while at the same time the performance 
of New Deal providers continues to improve ? 



i.2 Project aim 

The project aimed to identify the underlying causes for the worsening or 
improvement of grades, and whether these relate to individual providers 
or systems. 
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i.3 Method 



The investigation involved: 

■ a statistical analysis of the TSC and ALI inspection grades since 1998 

■ a qualitative review of inspection data 

■ consultation with the ALI and other stakeholders 

■ a survey of providers, local LSC quality managers and ALI inspectors. 

In-depth interviews were conducted with ALI inspectors, local LSC 
quality managers and a representative sample of training providers. 

The sample included providers whose grades have changed significantly. 

A standard interview tool (available on request) was used to investigate 
some of the possible causes of poor grades identified in the data analysis 
and discussions. 
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2 Findings 



2.1 Statistical analysis of inspection grades 

The research project analysed information on the inspection grades awarded 
by the TSC and ALI from April 1998 to December 2001. The analysis involved 
an examination of the overall grade profile, comparisons of grades awarded for 
leadership and management in ALI and its equivalent in TSC, and comparisons 
of TSC and ALI grades for areas of learning. Relationships between inspection 
grades and other factors, such as type and size of provider, were also undertaken. 
The analysis also considered data on achievement in government-supported 
training programmes. The commentary which follows notes significant findings 
from the full statistical analysis. 5 

2.1.1 All grades - work-based learning 

Figure 1 illustrates the distribution of grades in TSC and ALI inspections 
1998-2001. 

■ The proportion of grades 6 4 and 5 has increased from 18% in the first year of 
TSC and 23% averaged across 1998-2001 to 51% in the first 9 months of ALI. 
However, the number of providers inspected during the current year (232) is 
approximately 60% of the average of the three preceding years (378), and about 
two-thirds of the 347 conducted during 2000/01. (The number of inspections 
on which the comparisons are based is therefore lower in the current year. ) 

■ While the grade profile was deteriorating steadily in the last 2 years of TSC, 
the proportion of grades 4 and 5 has increased markedly under ALI. 

■ The proportion of grades 1 and 2 has fallen from 37 to 13% over the 4 years. 

■ The percentage of ‘satisfactory’ grades ( ie grade 3 ) has also declined from 45% 
in 1998/9 to 36% in 2001/02. 

■ The average grade has declined from 2.79 in 1998/9 and 2.89 across 
the total operating time of TSC to 3.46 under ALI. 

■ A quarterly analysis of grades indicates a ‘step change’ between TSC and ALI. 



5 Available from LSDA on request. 

6 In summarising their judgements about the quality of provision in curriculum or occupational areas 
(referred to as ‘areas of learning' (AoL) in inspection reports), and about the quality of leadership and 
management, ALI inspectors use a five-point scale. The descriptors for the five grades are : grade 1 - 
outstanding, grade 2 - good, grade 3 - satisfactory, grade 4 - unsatisfactory, grade 5 - very weak. 
These grades, and their descriptors, are similar to those used by the TSC. There are major differences, 
however, in the Common Inspection Framework (CIF), which is now used for work-based inspections, 
from that which the TSC used, as described in Raising the Standard (TSC 1998) . 
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Percentage Percentage 



Figure 1 Distribution of grades in TSC and ALI inspections 1998-2001 
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The numbers within the bars are the 
numbers of inspections on which 
the percentage is based 

The research only considered 
ALI inspections that were 
undertaken up to December 2001 
in the year 2001/02 
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2.1.2 New Deal inspection grades 



Average grades for all forms of New Deal provision have improved from TSC to 
ALI inspections, with the exception of the average grade for equal opportunities, 
which has worsened from 2.97 to 3.13 (see Table 1). 

Table 1 Comparison of New Deal average grades in TSC and ALI inspections 





TSC number 
of grades 


TSC average 


ALI number 
of grades 


ALI average 


Employment 


66 


3.06 


5 


2.60 


Self-employment 


53 


2.66 


2 


2.00 


Full-time education and training 


67 


3.34 


13 


3.15 


Environment Task Force 


67 


3.16 


10 


2.90 


Voluntary sector 


67 


2.94 


11 


2.55 


Leadership and management 


67 


3.39 


15 


3.27 


Equal opportunities 


67 


2.97 


15 


3.13 


Quality assurance 


67 


3.78 


15 


3.60 



2.1.3 Areas of learning 

Areas of learning were compared by matching as closely as possible the 
TSC’s occupational areas with the ALI’s areas of learning (AoLs ) . In this report, 
these are both referred to as areas of learning. 



Table 2 Areas of learning grades in TSC and ALI 



> 

o 

r~ 

o 

o 

cd Area of learning 



2 


Land-based provision 


29 


29 


38 


67 


38 


3 


Construction 


15 


17 


44 


59 


42’ 


4 


Engineering, technology and manufacturing 


11 


15 


19 


30 


15 


5/6 


Administration and ICT 


9 


15 


37 


46 


31 


7 


Retailing, customer service and transportation 


12 


17 


39 


51 


34 


8 


Hospitality, sports, leisure and travel 


12 


20 


45 


57 


37 


9 


Hairdressing and beauty therapy 


16 


21 


38 


54 


33 


10 


Health, social care and public services 


16 


21 


32 


48 


27 


11 


Visual and performing arts and media 








Low sample 


14 


Foundation programmes 


15 


16 
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23 


7 





— 1 
cn 
o 


o 

QJ_ 




CD 

CD 






> 

[Z 








sP 

O'" 






00 

S 






# 


—\ 

CO 






ora 


=T 




CO 






oa 


o 




=3* 


3 

Q_ 

CD 

C/3 


CD 
CD 
— H 

c n 


£ 


fl) 

=3 

CL 

M 


a 

=ti 

CD 


d’ 

ro 


03 

CL 

CD 

C/3 


03 
< 
CD 
— * 


a 

=p 

CD 


CD 


-P* 


o 


<D 
— * 


O 

O 


— * 
CD 


o 

o 


-p* 


0) 

oa 


CD 


CD 


0) 




03 




=3 




03 


CD 


=3 


00 

N 

CD 


=3 

CL 

CJl 


CD 
0) 
— * 
C/) 


ora 

CD 

3? 


N 

O 

ro 


O 

CD 

# 


s 

o 

ro 


=3 

a. 

CJl 


s 

> 

d 


O 

CD 



O 

ERIC 



19 



Findings 13 



Percentage 



As Table 2 shows, the area of learning grades show the same pattern 
of accelerated deterioration over the 4 years : 

■ the proportion of grades 4 and 5 has risen from 13 to 45% 

■ most areas of learning show a similar worsening 

■ foundation programmes are an exception : grades have deteriorated much less 
than the other areas of learning 

■ construction and hospitality, sports, leisure and travel grades have deteriorated 
more than average 

■ engineering, technology and manufacturing grades have deteriorated less 
than average. 

2.1.4 Leadership and management 

Leadership and management grades in ALI were first compared with the 
combined grades for management of training, equal opportunities and quality 
assurance in TSC, and later with management of training only. There were no 
significant differences in the results of these two comparisons. In this report, 
grades for these areas are referred to as leadership and management. 

Figure 2 Comparisons of TSC and ALI leadership and management grades 




0 TSC 1998/9 TSC 1999/2000 TSC 2000/01 ALI 2001/02 

Year of inspection 

□ G1 BG2 □ G3 □ G4 ■ G5 

Note : Numbers within or above the bars are the numbers of inspections on which the percentage 
is based 
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The examination of leadership and management grades revealed that : 

■ the leadership and management grades show the same pattern of 
accelerated worsening over the 4 years 

■ the grades are significantly worse for leadership and management than 
areas of learning 

■ the proportion of grades 4 and 5 is 56% for leadership and management in ALI 

■ the average grade has deteriorated from 2.97 to 3.59 over the 4 years, 
a drop of 0.62 of a grade 

■ no grade Is for leadership and management were awarded by ALI in 
the period considered by this project. 



2.1.5 Size of provider 

The influence of the size of provider on the grades awarded was investigated. 
The size of provider was estimated according to their numbers of learners, 
as shown inTable 3. 



Table 3 Leadership and management grades 4 and 5 awarded by size of provider 



Size of provider by % of combined grades 4 and 5 awarded 



number of learners 


1998/9 


1999/2000 


2000/01 


2001/02 


0-50 


26 


30 


31 


56 


51-100 


24 


32 


41 


49 


101-200 


32 


27 


48 


55 


201-500 


20 


29 


35 


47’ 


501+ 


20 


21 


38 


100 



The management and leadership grades for the providers in each band were 
examined. While the patterns are not uniform, there is a general increase in the 
percentage of grades 4 and 5 awarded since 1998/9. The 101-200 category 
saw fewer grades 4 and 5 between 1998/9 and 1999/2000, but all grades 
deteriorated in the last year of TSC and worsened still further under ALI. 
However, the distribution of the providers inspected was noticeably skewed 
in the current round of inspections towards those with fewer than 50 learners. 



2.1.6 Grade profiles by size of provider and specific area of learning 

Grade profiles in the three areas of learning with the most inspection grades 
(engineering, retailing and transportation, and business) were also considered 
by size of provider. 
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Figure 3a Average grades TSC versus ALI : retailing and transportation 




Figure 3b Average grades TSC versus ALI : engineering 
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Average grade awarded 



Figure 3c Average grades TSC versus ALI : business 




iiiiiiin 50 and under 1=1 51-100 ■ ■ 101-200 ■■ 201-500 o □ 501+ 



As Figure 3 reveals : 

■ there has been a decline in the average grade for all categories under ALI, 
except for providers with 51-100 learners in engineering, and those with 
101-200 in retailing and transportation 

■ the slight decline already apparent for retailing and transportation in 
the 50 learners or fewer category has sharply accelerated under ALI. 



2.1.7 Achievement in government-supported training programmes 

Achievement within all forms of work-based learning is difficult to judge, 
as the data available can be interpreted in several ways. 

Figure 4 shows the achievement rate as reported to DfES by 
government-supported training (GST) providers. This is based on providers’ 
responses to the leavers’ form which is returned to the DfES when trainees 
leave ( or complete ) the programme. This shows a relatively modest 
achievement rate. The highest achievement rate is 45%, and this includes 
both full and part NVQ achievement. These rates are likely to be better 
than the achievement of full Modern Apprenticeship frameworks, 
for which data is not available. 
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However, DfES notes that in 2000/01, 49% of Advanced Modern Apprenticeship 
(AMA) leavers achieved a qualification at Level 3 or above, and 45% of 
Foundation Modern Apprenticeship (FMA) leavers achieved a qualification 
at Level 2 or above. This is an improvement on the 22% achievement rate 
forAMAs in 1996/7, and the 36% rate for FMAs in 1999/2000. 



Figure 4 NVQ achievement rates in GST programmes by leaving year 




Table 4 shows completers, with part or full NVQs, as a percentage of the 
total leaving within the same year. 

Table 4 Achievement of part or full NVQs as percentage of completers with known outcomes 
by leaving year of trainee 7 



Programme 




1997/8 




1998/9 


1999/2000 




2000/01 


AMA 


23.6 


(25,069) 


38.6 


(47,288) 


44.8 


(64,226) 


44.1 


(59,860) 


FMA 


16.7 


(42) 


18.8 


(10,594) 


28.9 


(48,924) 


32.4 


(67,890) 


OTYP 8 


29.7 


(118,456) 


39.7 


(121,955) 


40.8 


(107,774) 


37.0 


(63,348) 


WBLA 8 


29.8 


(3297) 


29.2 


(72,141) 


25.1 


(101,201) 


16.4 


(102,809) 



Note: Numbers in brackets are actual numbers of completers on which the percentage is based 



7 The achievement rate has been calculated using standard LSC methodology : the numerator is 
[number of whole NVQ qualifications + (0.5 x the number of part NVQ qualifications)]; the denominator 
is the number of trainees for which NVQ qualification outcomes are known [whole + part + none ]. 

8 OTYP = other training for young people, WBLA = work-based learning for adults. 
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2. 1.7.1 Survey of trainees 



DfES also asks trainees about their achievement in a follow-up questionnaire, 
and the achievement rates they report are shown in Figure 5. This allows 
trainees to report NVQ achievement, which may not have been confirmed at 
the time of leaving the programme. 

Figure 5 NVQ achievement rates in GST programmes of ‘survey respondents’ by leaving year 




Table 5 indicates that NVQ achievement rates reported by trainees are generally 
higher than those reported by training providers, but it is important to note that 
these may be based on low survey response rates. The number of returns shown 
in Table 5 is a low percentage of the number of starts shown in Table 6, although 
the years of starting and completing vary and direct comparisons are therefore 
not possible. 

Table 5 Achievement rates reported by respondents to the DfES ‘follow-up’ survey 



Programme 




1997/8 




1998/9 


1999/2000 




2000/01 


AMA 


36.0% 


(4185) 


52.3% 


(9132) 


67.6% (11,629) 


54.9% 


(4902) 


FMA 


N/A 


(0) 


N/A 


(0) 


44.5% (4205) 


36.6% 


(5680) 


OTYP 


33.0% 


(30,368) 


44.2% 


(28,580) 


47.5% (27,684) 


24.9% 


(5171) 


WBLA 


42.7% 


(295) 


43.1% 


(24,304) 


46.0% (35,381) 


45.0% 


(29,826) 



Note : Numbers in brackets show the number of returns on which the percentage is based 

Non-completion does not necessarily mean that trainees have not achieved 
at all. There are also outcomes which are not directly related to qualifications ; 
for example, over a quarter of early leavers remain with the same employer who 
originally offered them the apprenticeship and about 20% go on to do similar 
work with a different employer. DfES follow-up studies suggest that satisfaction 
about the programme from both young people and employers is high. Inspection 
bases judgements about quality on completion of whole qualifications and 
full frameworks, as it is these that the provider is contracted to deliver. 
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2. 1.7.2 Decline in participation 



The number of starts in each programme for each of the 4 years has also changed 
considerably. As shown in Table 6, numbers appear to be declining, with the 
exception of FMAs, which may be increasing because of the phasing out of OTYP. 

Table 6 Number of starts on GST programmes 



April-September 



Programme 


1997/8 


1998/9 


1999/2000 


2000/01 


2001/02 


AMA 


87,864 


85,568 


86,522 


83,470 


33,400 


FMA 


1399 


41,043 


97,499 


106,052 


57,100 


OTYP 


204,321 


128,710 


71,548 


59,240 


24,600 


WBLA 


195,932 


101,126 


108,311 


108,772 


Not known 



2 . 1. 7. 3 Type of provider 

There are many different types of provider involved in work-based learning. 

The respective performance in leadership and management of the different types 
of provider is shown in Figure 6. 



Figure 6 Average leadership and management grades by type of provider 




Year of inspection 

""Min Private trainer cm Employer ■■ Charity /NFP m Local authority □ □ FE college 



9 One one FE college’s work-based learning provision was the subject of an ALI-led inspection 
during the period covered by the LSDA research. 
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As Figure 6 illustrates: 

■ grades for leadership and management have all worsened over the 4 years 
since 1998/9 

■ there are significant differences in the grades awarded to different types 
of provider 

■ there have been few grade Is awarded to charity /not-for-profit providers 
even though a relatively high number of inspections has been completed 

■ employer training providers were awarded more of the better grades for 
leadership and management and charity /not-for-profit providers received 
more of the poorer grades in this category. 

2.1.8 Providers inspected twice 

Up to the end of 2001, 20 work-based learning providers had been 
inspected twice, once byTSC in its first year of operation and once by ALI. 

The TSC inspection reports for these providers were published between 
22 July 1998 and 3 February 1999, and the last day of inspection by ALI 
ranged from 5 October to 14 December 2001. 

Comparison of the TSC and ALI grades indicates that: 

■ each of the providers’ average grades has worsened 

■ the proportion of grades 4 and 5 has risen from 7 to 59% 

■ in areas of learning, the proportion of grades 4 and 5 has risen from 7 to 56% 

■ in leadership and management, the proportion of grades 4 and 5 has risen 
from 10 to 65% 

■ the average decline for all grades is 0.97 of a grade 

■ areas of learning have declined by an average of 1.01 grades 

■ leadership and management grades have declined by an average 
of 0.90 of a grade. 

2.1.9 Grades for work-based learning provided by colleges 

At the time that the project research was undertaken, only a relatively small 
number of college (Ofsted-led) inspections had included work-based learning. 
Between September and December 2001, 38 college inspections took place, 

10 of which graded work-based provision. Twenty-three grades were awarded, 
with an average grade of 3.3. This compares with an average grade of 3.4 for 
the same areas of learning for non-college work-based learning (ALI inspections) 
over the same period of time. 

These grades were for areas of learning only, although they are likely to have 
contributed to leadership and management grades. In six cases, the grade 
awarded to the area of learning within which the college work-based learning was 
being inspected was the same, in four cases, the grade for work-based learning 
was better, and in 13 cases the work-based grade was worse. However, the 
college work-based learning provision has grades across 1-5, whereas the grades 
for other work-based training providers have no grade Is and more grade 4s. 

Of college work-based learning grades 57% were judged to be satisfactory or 
better compared with 49% of the grades awarded to other training providers 
for the same areas of learning ( see Figure 7 ) . 
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Percentage 



Figure 7( a ) Grades for college-led 
work-based training provision, 
inspected September-December 2001 
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Figure 7(b) Grades for other work-based 
training in the same areas of learning, 
inspected September-December 2001 



50 




0 1 2 3 4 5 

Grade 



Note : Numbers above the bars are the number of inspection grades on which the percentage is based 



2.2 Implications and issues arising from 
the statistical analysis 

The statistical analysis of work-based training providers confirms the 
acceleration under ALI of the deterioration of grades, which was evident 
in the latter part of the TSC regime. However, care must be taken in any 
conclusions derived from this evidence alone. There have been relatively 
few examples of the same providers experiencing both regimes, and the varied 
nature of the provider base and learners within it make comparing like with like 
an impossible task. The ALI data also covers less than a full year, and this has 
resulted in some of the numbers within the subgroups that have been examined 
(eg areas of learning) being quite small. The changes in emphasis and method 
of inspection from TSC to ALI need also to be taken into account. 

The different grade profile within New Deal provision further complicates 
the identification of underlying causes. Many of the issues encountered by the 
work-based learning providers are similar to those experienced by New Deal 
providers. In some cases, they may be the same providers, and the 
Common Inspection Framework (CIF) is used as the basis of ALI inspections 
of New Deal provision. 
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Despite these reservations, the marked deterioration in the grade profile in the 
20 providers that have experienced both ALI and TSC inspections suggests that 
there are significant underlying problems. Some of these may be connected with 
increasing robustness within the inspection process. Additionally or alternatively, 
ways of looking at evidence may be changing. The emphasis of the CIF on the 
impact of provision on learning within the CIF, as evidenced by achievement data, 
is likely to be having a significant effect on judgements about quality. 

The similarity in the average grade awarded for college-led work-based learning 
and other work-based provision suggests there are difficulties in delivering 
work-based learning across all types of providers. It perhaps also points 
to some inherent problems with the programmes on offer and the models 
of learning being used. 



2.3 Qualitative analysis of the ALI inspection reports 

The ALI inspection reports provide bullet points of key strengths and weaknesses. 
The ALI database, which identifies the number of times particular issues are 
mentioned, and the comments made, was examined to identify trends that 
could reveal the link between the grades awarded and the underlying concerns 
about the quality of practice. 

There is a very clear pattern of leadership and management issues raised 
in relation to areas of learning (seeTable 7). For all areas of learning, 
management of data was the most frequently mentioned weakness ( apart 
from in foundation for work, where it was staff development and appraisal ) . 

Poor analysis and use of data to guide decisions was mentioned twice as 
frequently as inadequate data or management information systems per se. 

The second most frequently mentioned weakness was internal verification 
(except in foundation programmes where it was leadership and strategy). 

The third issue most frequently mentioned was targets ( although management of 
learning was also the most mentioned weakness in six areas ) . Three-quarters 
of those weaknesses were in general setting of targets for improvement and 
staff performance, with a few specific references to retention and achievement 
and to under-represented groups. 

There was uniform ranking of leadership and management weaknesses 
mentioned under New Deal areas: management of data { 1), target-setting (2) 
and monitoring systems (3). 

A wide range of strengths and weaknesses are mentioned in inspection reports 
under areas of learning. As can be seen in Table 8, achievement and retention 
was the most frequently mentioned issue identified as a weakness, apart from 
in foundation programmes and foundation for work, where it was basic skills, 
and in hairdressing and beauty therapy where it was assessment processes. 
Second and third ranking issues were more widely distributed but the 
overall weaknesses in areas of learning are achievement and retention ( 1 ), 
assessment processes ( 2 ) and key skills ( 3 ) . 
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Achievement and retention was highlighted as a weakness 205 times. Of these 
141 were in areas graded 4 and 5 and 57 in areas graded 3. Only seven were 
mentioned in areas of learning gaining grade 2, and these were very wide-ranging, 
four comments relating to retention and the others to specific qualifications. 

An examination of comments in particular areas of learning such as foundation 
programmes and engineering did not show particular emphasis on either retention 
or achievement, although the number of providers in each area is small. 

Retention and achievement was also mentioned as a strength in 135 areas 
of learning/ New Deal areas, 84% of these in areas graded 3 or above, and 
only 16% in those graded 4 and 5 ( in New Deal grades 4 and 5 only 7%) . 

In work-based learning there were equal mentions of retention, achievement, 
and retention and achievement together in areas graded 1 or 2, whereas in 
areas graded 4 or 5, two-thirds of the mentions were for retention only. 

Assessment processes were recorded as a weakness in 159 areas of learning 
reports. Most referred generally to poor assessment practices, although there 
were also references to poor internal verification, training of assessors and 
planning of assessment. Most frequently mentioned ( 43 times ) was insufficient 
assessment in the workplace, also including lack of involvement of supervisors, 
employers and lack of assessment by direct observation. 

There were 139 mentions of key skills as a weakness. Specific weaknesses 
highlighted were poor or inadequate training, poor integration and/or late 
introduction of key skills, poor assessment practice, lack of understanding 
and poor progress by learners. 

The key areas of concern which are repeatedly highlighted in the ALI reports 
appear to be: 

■ management of data 

■ achievement and retention 

■ the quality of the assessment process. 

These issues broadly relate to those emerging from the quantitative analysis 
and the interviews with interested parties which follow. 
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Table 7 Strengths and weaknesses in leadership and management 
mentioned most frequently in inspection reports by areas of learning 
( l, 2 , 3 represent topics mentioned most frequently in rank order) 



Areas of learning 

Business administration 
Construction 
Engineering etc 
Foundation for work 
Foundation programmes 
Hairdressing and beauty 
Health etc 
Hospitality etc 
ICT 

Land-based 
Retailing etc 

New Deal 

Employment 

Environmental Task Force 

Full-time education and training 

Self-employment 

Gateway 

Voluntary sector 



Strengths 
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Table 8a Strengths in areas of learning mentioned most frequently in inspection reports 
( 1, 2, 3 represent topics mentioned most frequently in rank order) 
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Areas of learning 



Business administration 


Construction 


1 


2 




3 








Engineering etc 


3 


2 


1 










Foundation for work 




2 


3 






1 




Foundation programmes 


Hairdressing and beauty 


Health etc 


3 


1 


2 










Hospitality etc 




1 


=3 


=3 




2 




ICT 


2 


1 


=3 


=3 




=3 




Land-based 


1 


=2 


=2 




=2 






Retailing etc 


1 




=2 




=2 






AOLs overall 


1 


3 










2 


New Deal 
















Employment 


Environmental Task Force 


1 






=2 






=2 =2 


Full-time education and training 




2 




1 




3 




Self-employment 




=2 


1 






=2 


=2 


Gateway 


* 


1 


* 


* * 


* 


* 




Voluntary sector 


1 


2 






3 






New Deal overall 


2 


1 








3 




Overall 


2 


1 


3 











* All other areas equal 
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Table 8b Weaknesses in areas of learning mentioned most frequently in inspection reports 
( l, 2, 3 represent topics mentioned most frequently in rank order) 
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2.4 Consultation with ALI inspectors 

Members of the project team met with a group of approximately 60 
ALI inspectors to introduce the research project, ask for views on the reasons 
for the deterioration in grades and consult on the research instrument and 
sample of providers for the telephone interviews. The discussions took place 
in small groups and plenary sessions. 

2.4.1 Issues raised by inspectors 

■ The different focus of the CIF from Raising the Standard is perhaps best 
illustrated in the prominence of statistics on retention and achievement 
at the beginning of inspection reports. There is a strong emphasis in ALI 
on retention and achievement, and inspections are built around this. 

■ There is a need for caution in comparing the TSC and ALI grades ; we are not 
comparing like with like. The inspection framework has changed significantly. 
Inspectors commented that : 

□ while the emphasis now is on retention and achievement, 
providers may not always appreciate this 

□ expectations have increased, but providers have not 
been supported to raise their game 

□ inspectors have more data at their disposal, 
particularly on retention and achievement 

□ inspectors are better informed about programmes and qualifications. 

■ The emphasis on data about achievement and retention may also help to explain 
why New Deal grades have not declined in the same way. Data on outcomes, 
other than job outcomes, is not always available or used, as New Deal is 
primarily aiming to get clients into jobs and this is the most important measure 
of success. Achieving the full Modern Apprenticeship framework was seen to 
be more challenging. Achievement in key skills is often poor, which also impacts 
on grades awarded. The CIF provides less explicit examples for self-assessment 
than Raising the Standard. 

m There are changing expectations on what counts as completion in work-based 
learning- without a re-appraisal of the funding and resources to achieve this. 
Although more is expected of trainees, in terms of Modern Apprenticeship 
frameworks, but the time allowed for completion is not always sufficient. This 
is likely to be further exacerbated when key skills examinations come into play. 
Some questioned whether expectations were reasonable. 

■ Leadership and management grades are now more influenced by retention and 
achievement data. Even where training is good, if trainees do not successfully 
complete, provision is judged to be unsatisfactory, and this reflects in turn on 
leadership and management. 

■ Providers get less support for self-assessment from local LSCs than they 
did under the TECs. The reduction in support in the handover from TECs to 
local LSCs may have contributed to the problem but this did not fully account 
for the step change in deterioration of grades. 
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■ Providers are disadvantaged by not being predisposed to share practice 
and learn from each other. 

■ Re-inspections bring about improvements in outcomes and self-assessment, 
but the lasting effects of re-inspection are not known. 

■ The inspection focuses much more clearly on assessing provision from 
the learner's perspective - some inspectors also noted the CIF's emphasis 
on learning and the learning process. 

■ The breadth of inspectors' experience has increased, as have their expectations. 
Falling grades are not the result of poorer provision, but a reflection of higher 
expectations. Provision which was regarded as exemplary three years ago 
now looks more like the norm. Inspectors have got sharper and more confident. 
They are focusing on targets, the setting and achievement of targets, and 
continuous improvement. The training that inspectors have had on entry 

to ALI has developed the skills of new and experienced inspectors. 

These views provided valuable insight into inspectors’ perceptions of the 
reasons for poor quality in work-based learning and informed the areas 
of further investigation in the structured interviews. 



2.5 Views of established providers 

Two teleconferences were held with established providers to seek their views on 
the reasons for the deterioration of grades for work-based training. The issues 
emerging from the teleconferences indicated tensions between programme 
design, the ability levels of clients on these programmes, and capacity of the 
providers to deliver within the context of a more demanding inspection regime. 

2.5.1 Issues raised by established providers 

■ The focus on retention and achievement was thought to be a major contributory 
cause of declining inspection grades for work-based learning. Concerns were 
raised about the validity of the data on which judgements about retention and 
achievement are based, especially the use of historical information collected 
by TECs. The criteria for successful completion were perceived to have changed, 
from the TEC view that this was about achievement of NVQs to the achievement 
of broader Modern Apprenticeship frameworks. This was seen to limit the extent 
to which fair comparisons between TSC grades and ALI grades could be made. 

■ Teleconference participants noted that their trainees are not generally selected 
by ability. Indeed, if strict criteria for entry were adhered to, many providers would 
not be viable. Early leavers were thought to be caused by the poor guidance 
given before candidates are recruited, and also because young people in 
work-based training are ill equipped academically and attitudinally. Many trainees 
on work-based training programmes need a great deal of additional support. 

■ Changes to the CIF in relation to the absence of a specific quality rating for 
learner support and the lack of emphasis on developing basic skills were 
seen to be retrograde steps, which limited the extent to which the strengths 
of some providers could be celebrated. 
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■ Differences in approach and criteria for provider review and preparation for 
inspection, and lack of consistency in judgements between local LSCs, had 
been observed by teleconference participants. For example, in the case of a 
national provider, local LSC assessments of financial viability ranged from 
‘serious concern’ to ‘good’. 

■ General approval was expressed for the CIF’s emphasis on teaching and 
learning, but providers were concerned about the resource implications of this, 
especially for re-training of staff. 

■ Unease about the way grades are determined, and the perceived mismatch 
between the description of provision and the grade awarded, was expressed by 
some participants. The process appears subjective, and weighting for particular 
strengths and weaknesses unclear, and perhaps open to different 
interpretation by other inspectors. 

■ Early TSC inspections were seen to be a learning process for both inspectors 
and providers. Inspectors are now more confident, have seen more practice and 
can judge provision in relation to a range of other provision. 

■ Comparisons with FE college provision were seen to be at the heart of the 
‘secret’ ALI benchmarks, which providers firmly believe are being used by ALI. 

■ The arrangements for inspecting national providers were also commented upon. 
The time required for these inspections may prove to be onerous for providers, 
with some sites being inspected many times for different areas of learning. 

In large organisations, nominees will need to be released from all other duties. 

■ Funding was seen to have declined in real terms from TEC to LSC, the decline 
being estimated as between 15 and 25%. This reduction is being imposed within 
a more demanding regime, during a time of considerable change, alongside 

a fundamental shift from assessment-led provision to more direct training. 

There are concerns that the provider base is being reduced by attrition. 

■ The absence of sustained and sufficient support for staff development and 
training was seen as a major obstacle to improving provider performance. 

Staff recruitment and retention was quoted as a major inhibitor to quality 
improvement, especially as higher pay levels outside the work-based training 
sector result in qualified staff leaving to take posts in other sectors. 



2.6 Telephone interviews 

In order to investigate the concerns and issues raised by previous aspects of 
the research further, structured interviews were conducted with representatives 
of all the interested parties. These included a sample of providers, including 
providers previously inspected by TSC and ALI and those inspected only by ALI, 
ranging in size, type and geographical location. LSC quality managers and 
ALI inspectors, specialising in a wide range of areas of learning, were also 
interviewed. The interviews were conducted by telephone and each lasted 
approximately 45 minutes. 

Interviews were conducted with 18 providers inspected by both ALI and TSC, 

21 providers inspected by ALI only, 11 LSC quality managers and 
12 ALI inspectors. 
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2.6.1 Research instrument 



The research instrument for the telephone interviews included similar prompt 
questions for each type of participant, investigating the areas listed below. 

■ Quality and inspection : 

□ inspection framework 

□ the inspection process 

□ preparation and follow-up 

□ quality monitoring and improvement. 

■ Organisational Issues: 

□ stability and maturity of provider organisations 

□ scale and complexity of provision 

□ staff - roles, occupational competence, qualifications 

□ contracting and allocations arrangements. 

■ Leadership and management: 

□ management of the learning process 

□ self-assessment and development plans 

□ data 

□ strategic planning and quality assurance processes 

□ equal opportunities. 

■ The learning process: 

□ programmes and curriculum 

□ key skills 

□ teaching and learning 

□ retention and achievement. 



2.7 Findings from telephone interviews 

2.7.1 Quality and inspection 

2. 7.1.1 The inspection framework 

There was considerable agreement among LSC quality managers and 
ALI inspectors that the CIF places great emphasis on the impact of provision 
on the learner. 

The focus on experience of the learner has probably had most impact 
on grades. (LSC quality manager) 

The CIF focuses more on the learner and places more emphasis on 
retention and achievement. The TSC inspected the delivery of NVQs 
and the skills development of NVQs. The ALI places more emphasis 
on the teaching of theory and underpinning knowledge. (ALI inspector) 
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Some providers raised concerns that the attention paid to retention and 
achievement does not sufficiently take into account the type of learners 
and their various support needs. 

The more able trainees are remaining at school or college. Trainees who 
are coming into work-based learning have poorer basic skills. But as 
a company trainer, we can pick and choose our trainees. ( Provider) 

The extent to which providers understand the CIF varies greatly. Some noted 
that it is difficult to understand the principles before an inspection has taken 
place. Others noted that the CIF is less specific than Raising the Standard, 
and does not give as much guidance about what is expected. 

Raising the Standard was based on quality statements. It had the learner as 
a focus. The CIF is based on evaluative questions, which bring the learner 
even more to the fore. The CIF has a strong emphasis on teaching and 
learning as well as the learner. All its emphasis is on the impact on 
the learner. (LSC quality manager) 

Many providers noted the absence of explicit benchmarks for retention and 
achievement, and the lack of reference in expectations about retention to the 
prevailing levels of retention in jobs, as well as in training within occupational 
areas. The perception that there are ‘secret’ benchmarks and that these 
are unfairly drawn from college-based provision is firmly held. Some of those 
interviewed called for a review of the basis on which retention and achievement 
rates are calculated, and for recognition of the need to have an equitable 
measure of success across different post-16 learning routes. 



2. 7.1. 2 The inspection process 

Most providers agreed that the inspection process is fair, and great value 
is placed on the pre-inspection meetings with inspectors. While many 
were complimentary about the professionalism of the inspectors, a number 
expressed concerns about the attitude of some of the inspectors, and their lack 
of relevant experience in the occupational area or work-based training per se. 

Concerns were expressed about how grades were arrived at and the 
apparent contradiction between the commentary in inspection reports and the 
grades awarded. For some providers, this created anxiety about their efforts 
to improve. They are unclear about what needed to be done to improve the 
quality of what they do and are concerned that similar evidence may be 
judged differently by another inspector. 

Many providers stress that they put great effort into providing a good service, 
but find that their efforts are not acknowledged by the inspection process. 

The organisation is genuinely better than it was at TSC inspection but 
obtained worse grades. There is a lack of understanding at high levels 
as to how difficult work-based learning is. ( Provider) 

An ALI inspector observed that the TSC used to have grade descriptors 
and something like that could be useful. 



2. 7.1. 3 Preparation and follow-up 



Early ALI inspections appear to have suffered from inadequate support 
from local LSCs. There were examples given of providers basing their 
self-assessment on Raising the Standard, and reports that local LSCs were 
unable to provide answers to queries, as they, too, were coming to terms with 
the changes. However, many providers warmly praised post-inspection support 
from local LSCs. 

Providers noted the value of the self-assessment and development planning 
process. LSC quality managers saw some providers as paying lip service to 
these processes, and being driven by events, rather than anticipating changes 
in a strategic manner. The absence of a critical, self-improving culture was 
raised as a significant concern. The prevailing view is that providers generally 
do not know how to get better and need considerable support. 

ALI is spot on in its judgements. Providers are not analysing what they are doing. 
(LSC quality manager) 

Each provider has a contract manager who works with them on their 
self-assessment report and development plan and a quality manager. 

We provide workshops on common themes. It is difficult to know when to stop 
supporting and let them get on with it. Some national guidelines would help. 
(LSC quality manager) 



2. 7.1. 4 Quality monitoring and improvement 

Providers, and some LSC quality managers, noted the dissonance between 
the inspection and provider review processes. Provider review appears to be 
undertaken in different ways in local LSCs and ranges from a desk exercise 
to a full quality audit. 

Local LSCs and providers agree that the lack of sustained support for 
development and staff training within the work-based sector has been 
a major contributory factor in the failure to improve practice. 



2.7.2 Organisational issues 



2. 7.2.1 Stability and maturity of provider organisations and scale and complexity 

of provision 

There were conflicting views about the importance of the size and maturity 
of providers’ businesses and the scale of their provision. Some thought that 
small-scale provision could secure overall quality more easily. Others noted 
that there was a need for dedicated staff to deal with quality assurance and 
administrative issues, requiring economies of scale only possible in larger 
organisations. Inspectors commented that the broader the provision, the 
more scope there was for problems, although one observed that there 
could be an optimum size, below which providers struggle. 

Providers believe that companies with few vocational areas get better grades. 
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LSC quality managers, in particular, noted variations in the quality of provision 
across sites within national and regional providers. One quality manager 
particularly welcomed the ‘blitz’ approach to inspection of these providers. 

Quality can vary widely in different branches. Conformance to, and 
understanding of, quality assurance processes is variable. ALI’s ‘blitz’ 
approach in inspecting all sites is better. (LSC quality manager) 



2. 7.2. 2 Staff- roles, occupational competence and qualifications 

It is apparent that many providers operate on low staffing levels. Several 
providers mentioned the problem of recruiting and retaining staff, and their lack 
of resources to pay staff with appropriate qualifications was also of concern. 

Providers offering small minority areas of training find it difficult to 
obtain staff with the appropriate teaching qualifications or experience. 

(ALI inspector) 

Providers noted that they had put great effort and resources into training staff 
to be assessors, and now they needed to upskill them to become trainers. 

Some commented that once staff gained qualifications they left for jobs in 
other parts of the sector. 

Local LSC quality managers observed that some providers employed former 
trainees as trainers. Quality managers and inspectors saw what constitutes 
‘suitably competent and qualified’ as problematic. 



2. 7.2. 3 Contracting and allocations arrangements 

Most providers noted the changing nature of the client group with which they 
were dealing -their declining level of ability and motivation. Young people in 
the care sector, in particular, appear to be prone to leaving the profession to 
work in retailing, where pay and conditions are better. The value placed on the 
end qualification by these trainees appears to be low. 

At the same time, expectations of trainees have increased. Modern Apprenticeship 
frameworks are more rigorous than NVQs alone, and key skills are seen as 
a major obstacle to achievement. 

Given these problems with the client group, inadequacies in funding and 
the standard length of stay were seen as a contributory factor in unsuccessful 
completion. The lack of flexibility in the length of stay previously available in some 
TEC areas was seen to be a retrograde step. However, LSC quality managers 
noted the lack of awareness of providers about the potential for securing 
additional support for learners, either in-house or outsourced. 

Providers are crying out about revising the standard length of stay. 

There is confusion (possibly LSC's fault) about the availability of extra funding 
for additional learning needs. Use of this could resolve some of the issues. 
Support can be bought in from elsewhere. Because the providers don’t 
network, they probably would not know who could help. It is almost like 
they don't realise that there is a new system. (LSC quality manager) 
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There was general agreement that resources were insufficient to 
support successful completion and that funding had declined. 

For example, an Advanced Modern Apprenticeship in Business Administration 
last year was worth £2500. Now the same candidate would be worth £600. 
The emphasis of remuneration now is ‘bums on seats’ for a set amount of time. 
It's not worth doing. ( Provider) 

Some of the more successful providers interviewed had significant alternative 
resources fortheirtrainees - up to 75% extra. Providers noted that they usually 
supported the trainees to achieve their qualification aims outside the period 
of the standard length of stay - but this was not funded by the LSC. 



2.7.3 Leadership and management 

2. 7.3.1 Management of the learning process 

Providers noted that ALI appeared to expect to see management systems 
similar to those in colleges. Some providers observed that their processes 
were more informal, and less paper driven, but proving their effectiveness 
was more difficult in the absence of a ‘paper chase’. 

Some providers still experienced difficulties in managing subcontractors, but 
others had ended their contracts with other parties because of the difficulties 
in maintaining quality. 

Local LSCs noted the absence of a strategic view of the management of learning, 
and the tendency of small providers to be intimidated by larger organisations, 
usually colleges, with whom they subcontract. 

2. 7.3. 2 Self-assessment and development plans 

Most providers claimed to value the process of self-assessment and production 
of development plans, and saw them as useful. A minority of providers thought 
that these were produced solely for LSC/TECs. Some said they were using 
the CIF to drive the quality improvement process. 

2. 7.3.3 Data 

Providers reported many problems relating to the collection and interpretation 
of data. Many providers were concerned that historical data, collected for other 
purposes, was being used to judge the effectiveness of Modern Apprenticeship 
provision. Data collected for TECs was produced in different formats from that 
requested by ALI . Some TECs had judged successful completion on the basis 
of achievement of NVQs rather than completion of the Modern Apprenticeship 
frameworks. The lack of a standardised format for data collection by the LSC 
was of particular concern. Many providers were re-formulating their existing 
data at great cost, but were concerned that this would need to be re-done 
once LSC decided on a common format. 

Data issues related to the concerns expressed earlier over benchmarking 
were reiterated. 

Some inspectors noted the need to use data as a guide rather than 
an absolute measure. 

Inspectors need to consider the factors behind the data, 
rather than just considering raw data. (ALI inspector) 
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2. 7.3. 4 Strategic planning and the quality assurance process 

Providers were asked whether they were more concerned with contract 
compliance than quality improvement. Many were puzzled by this comparison 
and noted that they needed to comply with contracts in order to be given 
subsequent contracts. The concept of delighting rather than satisfying does 
not appear to be widely held, and reinforces earlier observations about the 
lack of strategic vision in many providers. 

LSC quality managers thought that providers lacked self-awareness and 
strategic vision. Providers were seen by some to do the minimum to meet 
requirements. This had engendered a dependency on the TEC, which was now 
continued into the LSC. Some inspectors also observed that providers did not 
exploit the CIF as a management tool - using it only to get through inspection. 

Inspection should be encouraging providers to become less reliant and 
more in charge of their own destiny, but this won’t happen overnight. 

It needs to be developed. (LSC quality manager) 

2. 7.3.5 Equal opportunities 

In the majority of cases, equal opportunity issues were not seen to have a 
detrimental impact on grades. Provision was said to be improving in many 
cases, although issues relating to occupational bias were raised. In general, 
it was thought that the CIF was more receptive than the TSC to the efforts 
of providers to overcome issues of bias. 

2.7.4 The learning process 

2. 7. 4.1 Programmes and curriculum 

The demands of the Modern Apprenticeship frameworks and the 
associated need for dedicated learning time were raised as major difficulties. 

This has to have an impact on judgements about quality because 
inspectors have to see teaching in action. (LSC quality manager) 

Some providers were especially concerned about the impact that technical 
certificates may have on the quality of provision once they are introduced. 
Some expressed concerns about the low tolerance of employers for providing 
time off to undertake specific off-the-job training sessions. There were 
also concerns about the extent to which trainees would accept a more formal 
delivery system, if this were to be similar to the formal, classroom-based 
system that many had disliked during compulsory schooling. 
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2. 7.4.2 Key skills 

Key skills were almost universally seen by those interviewed as a burden and 
barrier to achievement. The prospect of key skills tests was seen as a further 
contributing factor to decline in achievement. Many reported that trainees 
were refusing to engage in key skills sessions and that employers were 
supporting their trainees in this. The issue that was raised was not that 
key skills are irrelevant, but that there was a lack of flexibility in relating 
the key skills content to the main learning programme. 

The inability of providers to deliver key skills - either integrated into 
other learning or discretely - is seen by all concerned as a major obstacle 
to the development of high quality provision. 

2. 7.4.3 Teaching and learning 

Many providers welcomed the explicit reference to teaching and learning 
embodied in the CIF. However, some noted the need for a broader view of 
how this may be delivered in the workplace. Others were opposed to the 
notion of explicit teaching within work-based learning. The need for resources to 
upskill assessors to provide direct training was seen as an additional demand. 
Not all providers appreciated the significance of the switch from the 
NVQ assessment process to the delivery of Modern Apprenticeship programmes 
with greater need for underpinning knowledge and theory. Providers did not 
always see the relationship between better achievement and more rigorous 
teaching and learning. 

In contrast, inspectors emphasised the need for more defined teaching 
and learning activity. 



2. 7.4.4 Retention and achievement 

Most providers held the view that retention and achievement levels expected 
by ALI were unrealistic. Poor initial guidance and a lack of awareness of what 
the job entails was seen often to result in young people leaving within the 
first few weeks of a programme. Providers noted that, in contrast to colleges, 
they had few alternatives to offer when this was the case. Some also 
observed that four years ( in some cases ) was a long time in which to take 
a Modern Apprenticeship programme. Staging posts along the way may 
be helpful in maintaining motivation for young people. 

Inspectors were convinced that retention and achievement was a major indicator 
of the quality of provision, and that it was legitimate to expect that recruits have 
more than a 50% chance of successfully completing their training programmes. 



2.8 Triangulating issues from the structured interviews 

The structured interviews revealed similarities and differences in the 
most frequently mentioned concerns by providers, LSC quality managers and 
ALI inspectors. Table 9 triangulates the concerns most frequently mentioned 
by each of the interested parties. 
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Table 9 Triangulating concerns - priority rankings 10 



Issue identified 


Providers 


LSC quality managers 


ALI inspectors 


Retention and achievement 


=1 


5 


1 


Lack of support 


=1 




=3 


Unfamiliar with CIF 


=2 




=4 


Insufficient funding 


=2 






Key skills 


=3 


4 


=4 


Less able learners 


=3 


7 




Data 


4 


2 




Impact on learners and learning 




1 


2 


Providers are not strategic 




3 




Training programme issues 




6 


=3 



Size and scale of company 5 



These concerns indicate the different perspective on the problem 
as perceived by the key players. 

Providers : 

■ are aware of the impact of retention and achievement, but do not link this 
to impact on learners and learning 

■ are working with a new system of inspection which they do not fully understand 

■ lack support or resources to remedy their shortcomings 

■ have difficulty in delivering key skills 

■ think they are working with increasingly less able or motivated trainees 

■ have problems in obtaining and using data. 

LSC quality managers : 

■ have significant concerns about the impact of provision on learners and learning 

■ see data collection and interpretation as a key issue 

■ see providers as relatively immature organisations 

■ see key skills and training programme issues as part of the problem 

■ share to some extent the providers’ views about changes to the cohort 
of learners in the work-based route. 

ALI inspectors : 

■ closely link retention and achievement issues with impact on learners and learning 

■ share a similar priority ranking with providers regarding lack of support, 
and to a lesser degree with providers’ unfamiliarity with the CIF 

■ agree that there are key skills and other programme-related issues 

■ think there is some relationship to the size and scale of the organisation. 



10 The issues most frequently mentioned by each key player were not entirely consistent. 



38 Making the grade 



44 



3 Implications of findings 



The reasons for the decline in inspection grades for work-based learning 
appear to be multi-causal. Some of these may be related to transition from 
TEC to LSC, and as such may be transitory in their nature. However, the CIF and 
ALI inspection process may be revealing deeper systemic issues, which require 
attention from policy-makers if work-based learning is to improve. 



3.1 Understanding the CIF and support for inspection 

Lack of understanding and awareness of the new inspection framework on 
the part of the local LSCs and providers probably had a detrimental effect 
on providers’ readiness for inspection in the early days of ALI. Inadequate 
preparation for changes in self-assessment reports, data collection etc 
may have had some effect on grades. However, this appears to be being 
remedied in some local LSCs as they become more confident in their role. 
Post-inspection support is said to be good by many practitioners. 



3.2 Data and benchmarks for retention and achievement 

Many providers are convinced that there are ‘secret’ benchmarks being used 
by ALI, which are not related to occupational averages for either length of stay 
in certain jobs, or achievement of qualifications. Some maintain that there is 
an unfair comparison with further education colleges - in terms of the data 
on which retention and achievement is based. In further education colleges, 
achievement rates are calculated by expressing the number of learners 
obtaining their target qualification as a percentage of those who complete 
their learning programme. Retention rates are based on the numbers of 
learners completing their course as a percentage of those enrolled at the 
first census date . 11 In work-based learning the achievement rate is calculated by 
expressing the number of learners fully achieving all elements of their individual 
learning plan as a percentage of those who started the programme. The 
calculation in school sixth forms is estimated in yet another way. ALI’s view is 
that its expectation of average grades is influenced by the occupational area 
and programmes concerned, and not on comparison with college programmes. 

An explicit and shared basis on which judgements are made about retention 
and successful completion would be welcomed by all concerned. 



11 This date was November of the respective academic year, but is being changed to October in 2002. 
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Some providers noted that there were other outcomes in addition to the 
achievement of qualifications and completion of the Modern Apprenticeship 
framework which should be considered as measures of success. Aspects of 
provision, such as support for learners, are not inspected as a separate area 
in the CIF, although they were in Raising the Standard. Equally, one of the 
critical success factors in New Deal is the extent to which programmes can be 
adapted to meet the needs of individuals. There may be merit in reconsidering 
whether a range of value-added measures of success would be useful in 
estimating progression and development. 



3.3 Comparison with New Deal 

Some of the factors involved in the delivery and inspection of New Deal 
and work-based training are the same : 

■ the inspection framework is largely similar 

■ inspectors are drawn from ALI 

■ providers offering training for New Deal may also be offering work-based training. 
Differences in the programmes include: 

■ emphasis on job outcomes from New Deal has been a consistent policy objective 

■ definitions of achievement are more flexible than completion 
of a full Modern Apprenticeship framework with key skills 

■ provision is organised and inspected within a unit of delivery, which encourages 
collaboration and provides opportunities for networking and support 

■ Employment Services nationally have provided a substantial amount of 
guidance and have disseminated examples of good practice consistently 
since the early days of the programme. 

The support that providers receive and a more flexible view of achievement 
may make the task of delivering to a difficult client group less onerous. 

3.4 Providers or range of provision? 

The fall in quality reflected in the ALI grade profile appears in part to be related 
to the capacity of providers to deliver the more rigorous Modern Apprenticeship 
programmes, combined with the increased scrutiny of the impact of provision 
on learning within the CIF. 

The emphasis on a wider range of learning activities and more structured 
delivery of underpinning theory is a marked shift. The pedagogical implications 
of delivering Modern Apprenticeships need to be considered. Difficulties in 
delivering key skills are illustrative of the problems currently being faced by 
work-based training providers. The introduction of technical certificates is seen 
as a potential future problem, mainly because they will require more direct 
delivery of learning. 
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In some sectors,' providers are unsure about how to amend their practice 
to secure more regular and planned ‘learning episodes’, which are needed in 
programmes not exclusively leading to NVQs. Providers, employers and learners 
may not know what is required, and in some cases the changes may not 
be practical without a radical change in delivery and resourcing. 

Providers and LSC quality managers were concerned that there is not a 
good enough range of provision to meet the needs of all actual and potential 
learners in the work-based route. Providers report that in some instances 
they are encouraged to place trainees on Advanced Modern Apprenticeship 
programmes. As a result, young people are often placed on inappropriate 
programmes, which they subsequently leave without completing. This is 
a different, although related, issue from poor initial assessment. 

Providers see the emphasis in the CIF on the impact of provision on learners as 
a significant change. While AU inspectors noted that this emphasis was always 
a strong feature of Raising the Standard, it may be the most significant reason 
for providers’ difficulties in improving the quality of their provision. Providers 
see the unreliability of data on learners’ retention and achievement as a major 
contributory factor in the decline in grades. However, even at its most fragile 
the data reveals that people on work-based programmes are more likely to 
be unsuccessful than successful in achieving their learning goals. 

The focus on teaching and learning is a result of the switch from assessment 
of competence in role to the development of knowledge and skills in a work 
context. The work-based sector is geared up to deliver the former and will need 
support to find new ways of delivering effective work-based learning for people 
at the initial stages of their careers. These increasing demands do not appear 
to have been anticipated by work-based training providers. The support for 
their introduction has been patchy across TEC and now local LSC areas. 

The expertise within the work-based route clearly has many strength in terms 
of occupational expertise, and more recently in assessment processes. 
However, experience in more systematic teaching and learning in 
a work-based context is less well developed. 



.5 Provision for young people or adults? 

The inspections considered in this research project have almost entirely 
focused on training for young people at the formative stage of their career. 
The picture may be different for adults who are mature learners, for whom 
a work-based assessment of competence may be entirely adequate. 

In the case of one provider, the focus on Modern Apprenticeships resulted 
in very poor grades for only 3% of the total provision. There may be issues 
relating to young people and ‘pre-service’ learning, which is less problematic 
for adults in the workplace with significant experience that can be accredited 
and extended by means of work-based assessment. 
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3.6 Capacity building 

Many providers and LSC staff commented on the lack of sustained and 
sufficient development and training within the work-based training sector. 

The contrast with the college sector and its tradition for networking and 
exchanging practice was made, as was providers’ lack of access to 
the Standards Fund until very recently. 

The absence of a strategic view of many organisations offering work-based 
learning was also seen to contribute to the lack of responsibility taken by 
providers for their own development -to anticipate and prepare for change. 

The work-based sector is under-qualified, even in terms of the previous focus 
on assessment. Most providers interviewed had made great efforts to ensure 
that their staff obtained assessor qualifications and saw the cost of re-skilling 
assessors to become trainers as prohibitive. 



3.7 Key skills 

These are seen by the majority of providers as a major problem - not necessarily 
in their inclusion in vocational learning programmes, but in the relevance of 
the mandatory key skills, and their testing. Many providers note the difficulty in 
teaching key skills, and the danger that, if this is done in a formal, class-based 
manner, it could undermine the informal methods developed within work-based 
training, which are greatly appreciated by young people for whom school was 
a negative experience. 



3.8 Funding and resourcing issues 

Many providers commented on the reduction in real terms of the funding 
available to support work-based learning, and on the period of time during 
which funding is available. Problems with the standard length of stay and 
also local variations in this appear to be related to the increased demands of 
Modern Apprenticeships programmes, and the emphasis on higher achievement 
within the available resources. This problem is further exacerbated by the often 
modest entry qualifications of young trainees, and their subsequent need 
for significant additional support and extra time to achieve. 
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4 Recommendations 



The research findings have revealed a significant agenda for the improvement 
of work-based learning. This will require attention to immediate gains in quality 
improvement, as well as detailed consideration of some of the underlying 
issues which are hindering the development of a robust and effective 
work-based learning route for young people. 

■ In the short term, attention needs to be paid to the respective roles of 
the LSC staff and ALI inspectors in supporting providers. Greater clarity 
and transparency in expectations about success rates should be sought, 
and better and more comprehensive guidance on the CIF and its implications 
for self-assessment and inspection would be helpful to providers. 

■ Consideration should also be given to the range of programmes on offer to 
ensure that there is a sufficient breadth of provision to encourage participation, 
to sustain learners’ interest and secure progression. 

■ In the longer term, there is a need to build the capacity of work-based providers 
and also to develop new ways of delivering work-based learning for young people. 
This may require a substantial development programme, with exchange and 
development of practice. It would be beneficial if this process encouraged 
cross-fertilisation from all providers in the post-16 education and training sector, 
and also from other programmes, such as New Deal. 

■ Achievement in Modern Apprenticeships must be improved if the economic 
challenges of the new century are to be met. In addition to the development 
of better work-based learning, there is also a need to encourage more able 
young people to make apprenticeship their preferred route, rather than 

a second choice. 
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